Derivation and assessment of risk prediction models using case-cohort data

Background Case-cohort studies are increasingly used to quantify the association of novel factors with disease risk. Conventional measures of predictive ability need modification for this design. We show how Harrell’s C-index, Royston’s D, and the category-based and continuous versions of the net reclassification index (NRI) can be adapted. Methods We simulated full cohort and case-cohort data, with sampling fractions ranging from 1% to 90%, using covariates from a cohort study of coronary heart disease, and two incidence rates. We then compared the accuracy and precision of the proposed risk prediction metrics. Results The C-index and D must be weighted in order to obtain unbiased results. The NRI does not need modification, provided that the relevant non-subcohort cases are excluded from the calculation. The empirical standard errors across simulations were consistent with analytical standard errors for the C-index and D but not for the NRI. Good relative efficiency of the prediction metrics was observed in our examples, provided the sampling fraction was above 40% for the C-index, 60% for D, or 30% for the NRI. Stata code is made available. Conclusions Case-cohort designs can be used to provide unbiased estimates of the C-index, D measure and NRI

World Health Organization cardiovascular disease risk charts: revised models to estimate risk in 21 global regions

© 2019 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license Background: To help adapt cardiovascular disease risk prediction approaches to low-income and middle-income countries, WHO has convened an effort to develop, evaluate, and illustrate revised risk models. Here, we report the derivation, validation, and illustration of the revised WHO cardiovascular disease risk prediction charts that have been adapted to the circumstances of 21 global regions. Methods: In this model revision initiative, we derived 10-year risk prediction models for fatal and non-fatal cardiovascular disease (ie, myocardial infarction and stroke) using individual participant data from the Emerging Risk Factors Collaboration. Models included information on age, smoking status, systolic blood pressure, history of diabetes, and total cholesterol. For derivation, we included participants aged 40–80 years without a known baseline history of cardiovascular disease, who were followed up until the first myocardial infarction, fatal coronary heart disease, or stroke event. We recalibrated models using age-specific and sex-specific incidences and risk factor values available from 21 global regions. For external validation, we analysed individual participant data from studies distinct from those used in model derivation. We illustrated models by analysing data on a further 123 743 individuals from surveys in 79 countries collected with the WHO STEPwise Approach to Surveillance. Findings: Our risk model derivation involved 376 177 individuals from 85 cohorts, and 19 333 incident cardiovascular events recorded during 10 years of follow-up. The derived risk prediction models discriminated well in external validation cohorts (19 cohorts, 1 096 061 individuals, 25 950 cardiovascular disease events), with Harrell\u27s C indices ranging from 0·685 (95% CI 0·629–0·741) to 0·833 (0·783–0·882). For a given risk factor profile, we found substantial variation across global regions in the estimated 10-year predicted risk. For example, estimated cardiovascular disease risk for a 60-year-old male smoker without diabetes and with systolic blood pressure of 140 mm Hg and total cholesterol of 5 mmol/L ranged from 11% in Andean Latin America to 30% in central Asia. When applied to data from 79 countries (mostly low-income and middle-income countries), the proportion of individuals aged 40–64 years estimated to be at greater than 20% risk ranged from less than 1% in Uganda to more than 16% in Egypt. Interpretation: We have derived, calibrated, and validated new WHO risk prediction models to estimate cardiovascular disease risk in 21 Global Burden of Disease regions. The widespread use of these models could enhance the accuracy, practicability, and sustainability of efforts to reduce the burden of cardiovascular disease worldwide. Funding: World Health Organization, British Heart Foundation (BHF), BHF Cambridge Centre for Research Excellence, UK Medical Research Council, and National Institute for Health Research

Equalization of four cardiovascular risk algorithms after systematic recalibration: Individual-participant meta-analysis of 86 prospective studies

© 2018 The Author(s). Published by Oxford University Press on behalf of the European Society of Cardiology. Aims There is debate about the optimum algorithm for cardiovascular disease (CVD) risk estimation. We conducted head-to-head comparisons of four algorithms recommended by primary prevention guidelines, before and after \u27recalibration\u27, a method that adapts risk algorithms to take account of differences in the risk characteristics of the populations being studied. Methods and results Using individual-participant data on 360 737 participants without CVD at baseline in 86 prospective studies from 22 countries, we compared the Framingham risk score (FRS), Systematic COronary Risk Evaluation (SCORE), pooled cohort equations (PCE), and Reynolds risk score (RRS). We calculated measures of risk discrimination and calibration, and modelled clinical implications of initiating statin therapy in people judged to be at \u27high\u27 10 year CVD risk. Original risk algorithms were recalibrated using the risk factor profile and CVD incidence of target populations. The four algorithms had similar risk discrimination. Before recalibration, FRS, SCORE, and PCE over-predicted CVD risk on average by 10%, 52%, and 41%, respectively, whereas RRS under-predicted by 10%. Original versions of algorithms classified 29-39% of individuals aged ≥40 years as high risk. By contrast, recalibration reduced this proportion to 22-24% for every algorithm. We estimated that to prevent one CVD event, it would be necessary to initiate statin therapy in 44-51 such individuals using original algorithms, in contrast to 37-39 individuals with recalibrated algorithms. Conclusion Before recalibration, the clinical performance of four widely used CVD risk algorithms varied substantially. By contrast, simple recalibration nearly equalized their performance and improved modelled targeting of preventive action to clinical need

Assessing risk prediction models using individual participant data from multiple studies

Individual participant time-to-event data from multiple prospective epidemiologic studies enable detailed investigation into the predictive ability of risk models. Here we address the challenges in appropriately combining such information across studies. Methods are exemplified by analyses of log C-reactive protein and conventional risk factors for coronary heart disease in the Emerging Risk Factors Collaboration, a collation of individual data from multiple prospective studies with an average follow-up duration of 9.8 years (dates varied).We derive risk prediction models using Cox proportional hazards regression analysis stratified by study and obtain estimates of risk discrimination, Harrell’s concordance index, and Royston’s discrimination measure within each study; we then combine the estimates across studies using aweighted meta-analysis. Various weighting approaches are compared and lead us to recommend using the number of events in each study. We also discuss the calculation of measures of reclassification for multiple studies. We further show that comparison of differences in predictive ability across subgroups should be based only on within-study information and that combining measures of risk discrimination from casecontrol studies and prospective studies is problematic. The concordance index and discrimination measure gave qualitatively similar results throughout. While the concordance index was very heterogeneous between studies, principally because of differing age ranges, the increments in the concordance index from adding log C-reactive protein to conventional risk factors were more homogeneous.Lisa Pennells, Stephen Kaptoge, Ian R. White, Simon G. Thompson, Angela M. Wood and the Emerging Risk Factors Collaboration (Debbie A. Lawlor

Lifestyle index for mortality prediction using multiple ageing cohorts in the USA, UK and Europe

Current mortality prediction indexes are mainly based on functional morbidity and comorbidity, with limited information for risk prevention. This study aimed to develop and validate a modifiable lifestyle-based mortality predication index for older adults. Data from 51,688 participants (56% women) aged ≥50 years in 2002 Health and Retirement Study, 2002 English Longitudinal Study of Ageing and 2004 Survey of Health Ageing and Retirement in Europe were used to estimate coefficients of the index with cohort-stratified Cox regression. Models were validated across studies and compared to the Lee index (having comorbid and morbidity predictors). Over an average of 11-year follow-up, 10,240 participants died. The lifestyle index includes smoking, drinking, exercising, sleep quality, BMI, sex and age; showing adequate model performance in internal validation (C-statistic 0.79; D-statistic 1.94; calibration slope 1.13) and in all combinations of internal-external cross-validation. It outperformed Lee index (e.g. differences in C-statistic = 0.01, D-statistic = 0.17, P < 0.001) consistently across health status. The lifestyle index stratified participants into varying mortality risk groups, with those in the top quintile having 13.5% excess absolute mortality risk over 10 years than those in the bottom 50th centile. Our lifestyle index with easy-assessable behavioural factors and improved generalizability may maximize its usability for personalized risk management

Improving 10-year cardiovascular risk prediction in apparently healthy people : flexible addition of risk modifiers on top of SCORE2

AIMS: In clinical practice, factors associated with cardiovascular disease (CVD) like albuminuria, education level, or coronary artery calcium (CAC) are often known, but not incorporated in cardiovascular risk prediction models. The aims of the current study were to evaluate a methodology for the flexible addition of risk modifying characteristics on top of SCORE2 and to quantify the added value of several clinically relevant risk modifying characteristics. METHODS AND RESULTS: Individuals without previous CVD or DM were included from the UK Biobank; Atherosclerosis Risk in Communities (ARIC); Multi-Ethnic Study of Atherosclerosis (MESA); European Prospective Investigation into Cancer, The Netherlands (EPIC-NL); and Heinz Nixdorf Recall (HNR) studies (n = 409 757) in whom 16 166 CVD events and 19 149 non-cardiovascular deaths were observed over exactly 10.0 years of follow-up. The effect of each possible risk modifying characteristic was derived using competing risk-adjusted Fine and Gray models. The risk modifying characteristics were applied to individual predictions with a flexible method using the population prevalence and the subdistribution hazard ratio (SHR) of the relevant predictor. Risk modifying characteristics that increased discrimination most were CAC percentile with 0.0198 [95% confidence interval (CI) 0.0115; 0.0281] and hs-Troponin-T with 0.0100 (95% CI 0.0063; 0.0137). External validation was performed in the Clinical Practice Research Datalink (CPRD) cohort (UK, n = 518 015, 12 675 CVD events). Adjustment of SCORE2-predicted risks with both single and multiple risk modifiers did not negatively affect calibration and led to a modest increase in discrimination [0.740 (95% CI 0.736-0.745) vs. unimproved SCORE2 risk C-index 0.737 (95% CI 0.732-0.741)]. CONCLUSION: The current paper presents a method on how to integrate possible risk modifying characteristics that are not included in existing CVD risk models for the prediction of CVD event risk in apparently healthy people. This flexible methodology improves the accuracy of predicted risks and increases applicability of prediction models for individuals with additional risk known modifiers

Association between depressive symptoms and incident cardiovascular diseases

Importance: It is uncertain whether depressive symptoms are independently associated with subsequent risk of cardiovascular diseases (CVD). Objective: To characterize the association between depressive symptoms and CVD incidence across the spectrum of lower mood. Design, setting and participants: A pooled analysis of individual-participant-data from the Emerging Risk Factors Collaboration (ERFC; 162,036 participants; 21 cohorts; baseline surveys, 1960-2008; latest follow-up, March 2020) and UK Biobank (UKB; 401,219 participants; baseline surveys, 2006-2010; latest follow-up, March 2020). Eligible participants had information about self-reported depressive symptoms and no CVD history at baseline. Exposure: Depressive symptoms were recorded using validated instruments. ERFC scores were harmonized across studies to a scale representative of the Centre for Epidemiological Studies Depression scale (CES-D; range 0-60; ≥16 indicates possible depressive disorder). UKB recorded the Patient Health Questionnaire-2 (PHQ-2; range 0-6; ≥3 indicates possible depressive disorder). Main Outcomes and Measures: Primary outcomes were incident fatal/nonfatal coronary heart disease (CHD), stroke and CVD (composite of CHD and stroke). Hazard ratios (HRs) per 1-SD higher log-CES-D or PHQ-2 adjusted for age, sex, smoking and diabetes were reported. Results: Among 162,036 participants from the ERFC, 73% were female, mean (SD) age at baseline was 63 (9) years, and 5,078 CHD and 3,932 stroke events were recorded (median follow-up, 9.5-years). Associations with CHD, stroke and CVD were log-linear. HRs (95%CI) per 1SD higher depression score for CHD, stroke and CVD respectively were 1.07 (1.03-1.11), 1.05 (1.01-1.10), and 1.06 (1.04-1.08). This reflects, 36 versus 29 CHD events, 28 versus 25 stroke events, and 63 versus 54 CVD events per 1000 individuals over 10 years in the highest versus lowest quintile of CES-D (geometric mean CES-D score, 19 versus 1). Among 401,219 participants from the UKB, 55% were female, mean baseline age was 56 (8) years, and 4607 CHD and 3253 stroke events were recorded (median follow-up, 8.1-years). HRs per 1SD higher depression score for CHD, stroke and CVD respectively were 1.11 (1.08-1.14), 1.10 (1.06-1.14) and 1.10 (1.08-1.13). This reflects, 21 versus 14 CHD events, 15 versus 10 stroke events, and 36 versus 25 CVD events per 1000 individuals over 10 years in those with PHQ2 ≥4 versus 0. The magnitude and statistical significance of the HRs were not materially changed after adjustment for additional risk factors. Conclusions and Relevance: In a pooled analysis of 563,255 participants in 22 cohorts, baseline depressive symptoms were associated with CVD incidence, including at symptom levels below the threshold indicative of a depressive disorder. However, the magnitude of associations was modest.Lisa Pennells, Stephen Kaptoge and Sarah Spackman are funded by a British Heart Foundation Programme Grant (RG/18/13/33946). Steven Bell was funded by the National Institute for Health Research Blood and Transplant Research Unit in Donor Health and Genomics (NIHR BTRU-2014-10024). Tom Bolton is funded by the National Institute for Health Research Blood and Transplant Research Unit in Donor Health and Genomics (NIHR BTRU-2014-10024). Angela Wood is supported by a BHF-Turing Cardiovascular Data Science Award and by the EC-Innovative Medicines Initiative (BigData@Heart). John Danesh holds a British Heart Foundation Professorship and a National Institute for Health Research Senior Investigator Award.* *The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care

External validation, update and development of prediction models for pre-eclampsia using an Individual Participant Data (IPD) meta-analysis: the International Prediction of Pregnancy Complication Network (IPPIC pre-eclampsia) protocol.

Background: Pre-eclampsia, a condition with raised blood pressure and proteinuria is associated with an increased risk of maternal and offspring mortality and morbidity. Early identification of mothers at risk is needed to target management. Methods/design: We aim to systematically review the existing literature to identify prediction models for pre-eclampsia. We have established the International Prediction of Pregnancy Complication Network (IPPIC), made up of 72 researchers from 21 countries who have carried out relevant primary studies or have access to existing registry databases, and collectively possess data from more than two million patients. We will use the individual participant data (IPD) from these studies to externally validate these existing prediction models and summarise model performance across studies using random-effects meta-analysis for any, late (after 34 weeks) and early (before 34 weeks) onset pre-eclampsia. If none of the models perform well, we will recalibrate (update), or develop and validate new prediction models using the IPD. We will assess the differential accuracy of the models in various settings and subgroups according to the risk status. We will also validate or develop prediction models based on clinical characteristics only; clinical and biochemical markers; clinical and ultrasound parameters; and clinical, biochemical and ultrasound tests. Discussion: Numerous systematic reviews with aggregate data meta-analysis have evaluated various risk factors separately or in combination for predicting pre-eclampsia, but these are affected by many limitations. Our large-scale collaborative IPD approach encourages consensus towards well developed, and validated prognostic models, rather than a number of competing non-validated ones. The large sample size from our IPD will also allow development and validation of multivariable prediction model for the relatively rare outcome of early onset pre-eclampsia. Trial registration: The project was registered on Prospero on the 27 November 2015 with ID: CRD42015029349

Use of Repeated Blood Pressure and Cholesterol Measurements to Improve Cardiovascular Disease Risk Prediction: An Individual-Participant-Data Meta-Analysis

The added value of incorporating information from repeated blood pressure and cholesterol measurements to predict cardiovascular disease (CVD) risk has not been rigorously assessed. We used data on 191,445 adults from the Emerging Risk Factors Collaboration (38 cohorts from 17 countries with data encompassing 1962-2014) with more than 1 million measurements of systolic blood pressure, total cholesterol, and high-density lipoprotein cholesterol. Over a median 12 years of follow-up, 21,170 CVD events occurred. Risk prediction models using cumulative mean values of repeated measurements and summary measures from longitudinal modeling of the repeated measurements were compared with models using measurements from a single time point. Risk discrimination (Cindex) and net reclassification were calculated, and changes in C-indices were meta-analyzed across studies. Compared with the single-time-point model, the cumulative means and longitudinal models increased the C-index by 0.0040 (95% confidence interval (CI): 0.0023, 0.0057) and 0.0023 (95% CI: 0.0005, 0.0042), respectively. Reclassification was also improved in both models; compared with the single-time-point model, overall net reclassification improvements were 0.0369 (95% CI: 0.0303, 0.0436) for the cumulative-means model and 0.0177 (95% CI: 0.0110, 0.0243) for the longitudinal model. In conclusion, incorporating repeated measurements of blood pressure and cholesterol into CVD risk prediction models slightly improves risk prediction

Estimating individual lifetime risk of incident cardiovascular events in adults with type 2 diabetes: an update and geographical calibration of the DIAbetes Lifetime perspective model (DIAL2)

Background: The 2021 ESC cardiovascular disease (CVD) prevention guidelines recommend the use of (lifetime) risk prediction models to aid decisions regarding intensified preventive treatment options in adults with type 2 diabetes, e.g. the DIAbetes Lifetime perspective model (DIAL model). The aim of this study was to update the DIAL-model using contemporary and representative registry data (DIAL2) and to systematically calibrate the model for use in other European countries. Methods and Results: The DIAL2 model was derived in 467,856 people with type 2 diabetes without a history of CVD from the Swedish National Diabetes Register, with a median follow-up of 7.3 years (IQR 4.0-10.6 years) and comprising 63,824 CVD (including fatal CVD, nonfatal stroke and nonfatal myocardial infarction) events and 66,048 non-CVD mortality events. The model was systematically recalibrated to Europe’s low and moderate risk region using contemporary incidence data and mean risk factor distributions. The recalibrated DIAL2 model was externally validated in 218,267 individuals with type 2 diabetes from the Scottish Care Information – Diabetes (SCID) and Clinical Practice Research Datalink (CPRD). In these individuals, 43,074 CVD events and 27,115 non-CVD fatal events were observed. The DIAL2 model discriminated well, with C-indices of 0.732 (95%CI 0.726-0.739) in CPRD and 0.700 (95%CI 0.691-0.709) in SCID. Interpretation: The recalibrated DIAL2 model provides a useful tool for the prediction of CVD-free life expectancy and lifetime CVD risk for people with type 2 diabetes without previous CVD in the European low and moderate risk regions. These long-term individualized measures of CVD risk are well suited for shared decision making in clinical practice as recommended by the 2021 CVD ESC prevention guidelines